1. Technical Field
The present invention relates to image processing, more specifically to a method and a device that reduce delay of processing an image.
2. Description of the Related Art
By mounting a small or thin imaging device on a small or thin portable terminal, such as a portable phone or a PDA (personal digital assistant), the portable terminal can now function as an imaging device also. Thanks to this new development, the portable terminal can send not only audio information but also visual information. The imaging device has been also mounted on a portable terminal such as the MP3 player, besides the portable phone and PDA. As a result, a variety of portable terminals can now be embodied to capture an external image and retain the image as electric data.
Generally, the imaging device uses a solid state imaging device such as a CCD (charge-couple device) image sensor and a CMOS (complementary Metal-oxide semiconductor) image sensor.
FIG. 1 is a simplified structure of a typical imaging device, and FIG. 2 shows signal types for outputting encoded data. FIG. 3 shows the steps of a typical JPEG encoding process.
As shown in FIG. 1, the imaging device, converting the captured external image to electric data and displaying the image on a display unit 150, comprises an image sensor 110, an image signal processor (ISP) 120, a back-end chip 130, a baseband chip 140 and the display unit 150. The imaging device can further comprise a memory, for storing the converted electric data, and an AD converter, converting an analog signal to a digital signal.
The image sensor 110 has a Bayer pattern and outputs an electrical signal, corresponding to the amount of light inputted through a lens, per unit pixel.
The image signal processor 120 converts raw data inputted from the image sensor 110 to a YUV value and outputs the converted YUV value to the back-end chip 130. Based on the fact that the human eye reacts more sensitively to luminance than to chrominance, the YUV method divides a color into a Y component, which is luminance, and U and V components, which are chrominance. Since the Y component is more sensitive to errors, more bits are coded in the Y component than in the U and V components. A typical Y:U:V ratio is 4:2:2.
By sequentially storing the converted YUV in FIFO, the image signal processor 120 allows the back-end chip 130 to receive corresponding information.
The back-end chip 130 converts the inputted YUV value to JPEG or BMP through a predetermined encoding method and stores the YUV value in a memory, or decodes the encoded image, stored in the memory, to display on the display unit 150. The back-end chip 130 can also enlarge, reduce or rotate the image. Of course, it is possible, as shown in FIG. 1 that the baseband chip 140 also receives from the back-end chip 130, and display on the display unit, the decoded data.
The baseband chip 140 controls the general operation of the imaging device. For example, once a command to capture an image is received from a user through a key input unit (not shown), the baseband chip 140 can make the back-end chip 130 generate encoded data corresponding to the inputted external image by sending an image generation command to the back-end chip 130.
The display unit 150 displays the decoded data, provided by the control of the back-end chip 130 or the baseband chip 140.
Signal types during the steps of sequentially inputting the data, stored in the memory, to process, for example, decoding are shown in FIG. 2. Generally, the back-end chip 130 is realized to receive the YUV/Bayer-format data, and the P_CLK, V_sync, H_REF and DATA signals, are used as the interface for receiving this kind of data.
As shown in FIG. 2, since the conventional back-end chip 130 is designed to be inputted with a vertical synchronous signal (V_sync2) for a following frame (e.g. a (k+1)th frame) and/or image data while processing the encoding for one frame (e.g. a kth frame), there has been much possibility of causing an error during the image data encoding.
As a result, the conventional back-end chip 130 sometimes encodes not only the frame that is currently being processed but also the next frame, not completing the encoding of correct data.
Besides, when an encoding unit of the back-end chip 130 transfers encoded data to the decoding unit or stores the encoded data in the memory, the encoded data for the current frame is sometimes not inputted correctly after a new vertical synchronous signal (V_sync2) is received.
In addition, as shown in FIG. 2, the conventional image signal processor 120 alternately outputs the H_REF signal that may be used when the back-end chip 130 stores the data. This can cause power consumption due to switching of a write enable signal in the memory of the back-end chip 130.
FIG. 3 illustrates the steps of typical JPEG encoding, carried out by the back-end chip 130. Since the JPEG encoding process 200 is well-known to those of ordinary skill in the art, only a brief description will be provided here.
As illustrated in FIG. 2, the image of the inputted YUV values is divided into a block in the size of 8×8 pixels, and in a step represented by 210, DCT (discrete cosine transform) is performed for each block. DCT means a series of processes to perform Fourier transform by converting an image into a corresponding frequency in a space image.
Then, in a step represented by 220, a quantizer quantizes a DCT coefficient of each block by applying a weighted value according to the effect on the visual. A table of this weighted value is called a “quantization table.” A quantization table value takes a small value near the DC and a high value at a high frequency, keeping the data loss low near the DC and compressing more data at a high frequency.
Then, in a step represented by 230, the final compressed data is generated by an entropy encoder, which is a lossless coder. The quantized coefficients consist of a series of successive zeros. Accordingly, the data can be decreased in size through a run-length encoding that represents data as a code symbol. Also, through a Huffman encoding, the data can be decreased in size. The Huffman algorithm assigns a shorter symbol to the code symbol that is statistically more frequently repeated. Both run-length and Huffman encodings do not cause data loss while performing the encoding to reduce the data size.
The data encoded through the above steps is stored in a memory. The back-end chip decodes the data loaded in the memory and displays the data on the display unit 150 for example.
The typical image signal processor 120 is realized to operate in a clock rate lower than a clock rate of the back-end chip 140.
Generally, image sensor 110 outputs raw data in a Bayer image format. The outputted raw data is converted into RGB data through a predetermined pre-process (e.g. an image process or interpolation). The RGB data is converted into YUV data for outputting. As a result, the amount of outputted data becomes about twice as much as the amount of inputted data. Accordingly, the clock rate used in an output part of the image processor 120 becomes approximately twice as much as the clock rate used for image processing inside of the image signal processor 120. Similarly, the clock rate in the back-end chip 130 shall become approximately twice as much as the clock rate used for the image processing inside the image signal processor 120.
As described above, since the output part of the image signal processor 120 uses a high clock rate, there have been errors during the image processing.
With the recent trend toward higher resolution, the image processing speed has become an important factor. For example, a clock frequency of about 20 to 40 MHz used to be sufficient to render a satisfactory image when processing a conventional VGA image or a one-mega-byte image. As today's images often have 3 to 5 mega bytes or more, however, clock frequencies of 60 to 80 MHz are now required to render 10 to 20 frames per second.
Considering the conventional linked structure between the image signal processor 120 and the back-end chip 130, increasing the clock rate for image processing of the image signal processor 120 requires an increased clock rate of the back-end chip 130, which is technologically impossible.
As such, since the conventional image signal processor 120 processes a high resolution image with a low frequency clock rate, it takes more time to capture a scene. Moreover, the scene is captured not at once but with delay. This lagged capturing of the image causes distortion of the captured image of a moving object.
Besides, since the image processing is performed with a low frequency clock rate, image data of a following frame inputted before JPEG encoding of an image is completed causes a conflict.